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in  Readiness  Measurement 


/ 8STRACT 

This  report  develops  models  which  can  improve  the  readi¬ 
ness  measurement  system  by  focusing  on  the  use  of  the  commander's 
actual  readiness  evaluations  of  his  ship  to  measure  his  attitudes 
towards  risk  and  uncertainty.  It  is  concerned  with  readiness  for 
individual  missions  as  well  as  the  overall  readiness  of  the  unit. 
Some  conceptual  models  of  readiness  measurement  or  evaluation 
which  have  the  potential  for  improving  planning,  communication, 
control,  and  prediction.  These  models  are  developed  to  reveal 
the  relationships  among  the  relevant  elements  in  the  evaluation 
process  and  to  provide  a  framework  for  empirically  determining  a 
commander's  utility  functions  from  his  actual  readiness  evalua¬ 
tions.  Some  approaches  are  considered  for  using  these  models 
for  estimating  utilities  and  for  predicting  ship  o^rformance. 
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INTRODUCTION 


Some  of  the  rationales  for  using  utility  functions  to 
measure  commanders'  attitudes  towards  risk  and  uncertainty  as 
an  aid  in  the  readiness  measurement  problem  were  developed  in 
Technical  Report  No.  12.  Interview  approaches  have  been  used 
in  ~K>st  studies  which  have  been  conducted  of  attitudes  towards 
risk  and  uncertainty,  and  an  interview  approach  using  mission 
scenarios  and  responses  of  the  commander  to  choice  questions  in 
the  specified  circumstances  was  presented  in  the  aforementioned 
report.  The  scenarios  are  an  attempt  to  reflect  real  circum¬ 
stances  and  the  utility  functions  resulting  from  this  procedure 
can  be  informative  and  useful. 

However,  these  interview  procedures  can  be  somewhat 
impeded  by  the  fact  that  the  commander  does  not  have  to  bear 
the  consequences  of  his  readiness  evaluation  regardless  of  the 
realism  of  the  scenario.  For  this  reason,  some  people  have 
suggested  that  decisions  made  under  real  operating  conditions 
may  be  more  revealing  of  a  person's  attitude  towards  risk  and 
uncertainty. 

The  purpose  of  this  report  is  to  develop  models  which  can 
improve  the  readiness  measurement  system  by  focusing  on  the  use 
of  the  commander's  actual  readiness  evaluations  of  his  ship  to 
measure  his  attitudes  towards  risk  and  uncertainty.  It  is  con¬ 
cerned  with  readiness  for  individual  missions  as  well  as  the 
overall  readiness  of  the  unit. 
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The  evaluation  process  is  viewed  both  from  a  conceptual 
and  an  empirical  point  of  view.  A  commander's  evaluation  of  his 
ship  is  based  on  objective  data  regarding  material,  personnel, 
maintenance,  etc.  as  well  as  the  cow&ander's  knowledge,  exper¬ 
ience,  and  attitude  towards  risk  and  uncertainty  (his  utilities). 

We  present  in  this  report  some  conceptual  models  of 
readiness  measurement  or  evaluation  which  have  the  potential  for 
improving  planning,  communication,  control,  and  prediction. 

These  models  are  not  intended  to  be  descriptions  of  how  comman~ 
ders  rate  a  ship's  readiness  or  of  how  they  should  rate  the  ship. 
Rather  these  models  are  developed  to  reveal  the  relationships 
among  the  relevant  elements  in  the  evaluation  process  and  to 
provide  a  framework  for  empirically  determining  a  commander's 
utility  function  from  his  actual  readiness  evaluations. 

We  then  consider  some  approaches  for  using  these  models  for 
estimating  utilities  and  for  predicting  ship  performance. 


GENERAL  MODEL 

We  assume  in  our  general  model  that  the  ship  may  be  used 
for  various  missions,  which  are  denoted  l,2,...,m.  The  proba¬ 
bilities  that  the  ship  will  be  called  upon  to  perform  each  mission 
are  denoted  q1 ,q2# . . . ,qm  (qj  >  0?  I  q^  =  l). 

Performance  on  missions  is  affected  by  various  factors 
such  as  materiel,  personnel,  maintenance,  etc.  These  are 
denoted  l,2,...,f.  The  relative  importance  of  each  factor  for 
each  mission  may  vary  and  is  denoted  V^,  the  relative  importance 
of  factor  i  for  mission  j  (V^  >_  0;  l  *  1  for  j  =  1,2, . . .  ,ra) . 
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The  commander  is  assumed  to  evaluate  the  readiness  of 
factor  i  to  perform  in  mission  j.  This  evaluation  is  denoted 


This  conceptual  model  can  be  represented  by  the  following 

array: 


1 

2 

Factors 

i 


Missions 


The  estimate  of  the  overall  readiness  of  the  ship  can  be 
made  using  these  elements  in  a  number  of  different  ways*  Two  of 
the  possible  ways  for  estimating  overall  readiness ,  denoted  R*, 
are: 
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2.  Minimum  rating  or  weakest  link  method 

R*  *  Minimum  R., 

<i,j>  13 

SPECIAL  CASE  OF  ONE  FACTOR  AND  ONE  MISSION 

We  will  next  consider  in  some  detail  the  readiness  evalua¬ 
tion,  R  ^  for  one  particular  factor  i  and  one  particular  mission 
j- 

Let  the  possible  values  of  R_  be  1,2,. ..,R.  It  is 
assumed  that  there  is  an  actual  state  of  readiness*  or  level  of 
performance  which  is  obtained  from  people  other  than  the  commander. 

The  various  possible  actual  states  of  readiness  or  levels 
of  performance  are  labeled  1,2,...,S.  In  the  traditional  decision 
theory  framework,  these  states  of  readiness  or  performance  levels 
are  called  states  of  nature. 

R  is  usually  equal  to  S,  although  it  need  not  be. 

In  making  his  readiness  evaluation,  the  commander  chooses 
a  readiness  rating,  r,  from  1,2,...,R.  The  consequences  or 

* 

In  practice,  the  actual  state  of  readiness  is  presumably 
estimated  by  evaluators  based  upon  performance  (in  maneuvers  or 
in  combat)  or  by  higher-level  officers  who  review  the  factors 
affecting  the  readiness  of  the  ship.  These  officially  determined 
"actual"  states  of  readiness  are,  of  course,  affected  by  the 
utilities  or  attitudes  towards  risk  and  uncertainty  of  the 
evaluators  and  reviewers. 
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effects  of  this  choice  on  the  commander  depend  on,  among  other 
things,  the  actual  state  of  readiness  1,2,...,S). 

How  does  the  commander  choose  an  r?  In  this  model  we 
postulate  that  his  observations,  knowledge,  and  experience  provide 
one  aspect  of  his  decision  and  his  attitude  towards  risk  and 
uncertainty  provides  a  second  aspect.  He  makes  various  obser¬ 
vations  on  his  ship  and,  based  on  his  knowledge  and  previous 
experience,  estimates  the  probabilities  of  the  different  possible 
actual  states  of  readiness.*  These  probability  estimates,  which 
are  not  necessarily  made  consciously,  are  denoted  Pj/P2'***'Ps 
(p  il  0;  l  p  =1).  His  attitude  towards  risk  and  uncertainty  are 


summarized  by  his  utilities,  U which  denote  the  utility  of 
readiness  rating  r  when  the  actual  rating  is  s. 

The  situation  is  represented  in  the  following  arrays 

Commander’s  Readiness  Rating 


Actual 

State 

of 

Readiness 

(State  of 
Nature) 


It  could  be  argued  that  the  commander's  attitude  towards  risk  and 
uncertainty  might  also  exert  some  influence  on  Ms  estimates  of  these 
probabilities,  but  this  effect  should  be  of  secondary  importance. 


The  rating  that  is  used  can  now  be  computed  by  choosing 

S 

that  rating  r  which  will  give  the  largest  value  of  l  p  u_  . 

s«l  8  sr 

Some  important  statistical  questions  relating  to  this 
model  are: 

1.  Given  past  experience  with  a  commander's  readiness  ratings 
and  the  actual  states  of  readiness  on  n  evaluation  occasions 

*rl'sl* '  <r2's2* '  •  *  • '  *rn'sn* 9  how  can  Usr  an<*  P®rliaPs 
p's  be  estimated? 

2.  Given  past  experience  on  n  occasions  and  a  commander's 
readiness  rating  rn+1,  for  the  n+lth  evaluation  occasion , 
how  should  the  actual  state  of  readiness,  sf  fi,  be  predicted? 
In  other  words,  the  problem  is  to  predict  the  actual  state  of 
readiness  from  past  experience  with  a  commander  and  his 
readiness  rating. 

AGGREGATED  MODEL 

In  this  section,  a  less  detailed  model  is  presented. 

Here  the  different  missions  and  different  factors  are  not 
explicitly  stated.  (They  are,  of  course,  considered  by  the 
commander  in  making  his  readiness  evaluations.)  The  possible 
overall  readiness  ratings  of  the  ship  are  labeled  r*l,2,...,R. 
The  actual  performance  ratings  are  given  by  s*l,2,...,S.  As 
in  the  previous  model,  the  commander  estimates  the  probabilities 
of  the  actual  states  of  readiness  by  p1#p2,...,ps  from  his 
observations  on  the  ship  and  his  knowledge  and  experience. 
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His  utilities  are  denoted  by  Wgr  in  this  aggregated  model.  The 
model  can  again  be  presented  in  an  array: 

ander's  Readiness  Rating 

2  ...  r  .  .  .  R 

1 


W 

sr 


The  difference  be  ween  the  model  and  the  one  in  the 
previous  section  is  that  all  the  factors  and  missions  are  aggre¬ 
gated  here.  The  model  postulates  that  the  commander  chooses  the 

readiness  rat.ng  r  which  maximizes  £  Pewe^* 

s 

The  same  kinds  of  statistical  questions  arise  relating 
to  this  model  as  to  the  previous  disaggregated  one:  How  can  the 
commander's  utilities,  W  ,  and  perhaps  the  p's,  be  estimated? 

How  can  actual  states  of  readiness  be  predicted  from  the  comman¬ 
der's  readiness  rating? 

The  further  development  of  these  types  of  models  could  be 
helpful  in  understanding  the  dynamics  of  the  readiness  measure¬ 
ment  process  so  that,  for  example,  we  would  know  the  sensitivity 
of  the  system  to  errors  in  the  different  parameters  and  variables. 
This  could  be  valuable  for  designing  improvements  of  the  proce¬ 
dures.  For  example,  it  is  possible  that  time  and  effort  could  be 


Comm 


Actual 

State 

of 

Readiness 

(State  of 
Nature) 


Pi 1 

P,  2 


PS  S 
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spent  on  collection  of  data  which  could  yield  relatively  larger 
effect  on  the  final  readiness  measures  than  the  current  allo¬ 
cation  provides. 


ILLUSTRATIONS  OF  APPROACHES  AND  PROBLEMS 

To  illustrate  the  approaches  and  problems,  we  will  now 
discuss  methods  for  estimation  and  prediction  using  the  aggre¬ 
gated  model  as  our  example.  We  will  use  R=S=2.  There  are  two 
states  of  nature  (actual  states  of  readiness):  s  =  l  (satisfactory) 
and  s  =  2  (unsatisfactory)  as  well  as  two  ratings  for  the  commander 
of  the  ship,  r  =  l  (satisfactory)  and  r=z  (unsatisfactory).* 

The  situation  is  represented  by  a  square  array: 


n 

At  the  present  time,  the  Navy  uses  a  system  for  its  C-ratings 
in  which  R=S  =  4.  1  indicates  the  most  ready  condition  and  4 

indicates  the  least  ready.  We  have  been  told  that  a  large  propor¬ 
tion  of  all  commander  ratings  are  either  2  or  3,  with  ratings 
of  1  and  4  being  used  relatively  i.nrreguently.  If  one  wanted  to 
apply  the  current  ratings  to  our  simplified  example,  C-  1  and 
C-2  ratings  could  be  considered  satisfactory  (r  =  l)  and  C-3 
and  C-4  could  be  considered  unsatisfactory  (r  =  2).  (Or, 
depending  on  the  actual  usage  of  the  various  C-ratings,  C-l, 

C-2,  and  C-3  could  be  considered  satisfactory  (r=l)  and  only 
C  -  4  be  considered  unsatisfactory.) 
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Actual 

State  of 

Readiness 

(State  of 
Nature) 

The  values  of  and  are  the  numericai  assessments  of  the 

seriousness  of  situations  (s  =  2,  r  =  1)  and  (s  =  1,  r  =  2)  respectively. 
When  s  =  r,  a  zero  cost  is  assumed.  These  costs  can  be  thought  of 
as  negative  utilities. 

As  previously  discussed,  the  commander’s  choice  of  P 
depends  on  a  combination  of  his  knowledge  and  experience  with  his 
observations  of  the  ship  and  his  choice  of  values  for  and 
depends  on  his  utilities. 

The  commander’s  procedure  for  choosing  a  rating  is  to  j  ick 

r=l  when  (1  -  P)C21  <_  PC12  and  r  =  2  otherwise.  The  previous 

inequality  is  equivalent  to  choosing  r=l  when 

C  C 

21  21 

P  - .  -  will  be  denoted  by  c. 

C12+C21  C12+C21 


Commander’s  Readiness  Rating  (r) 


1-P 


'12 


'21 


Thus,  the  commander  rates  the  ship  as  satisfactory  (r =  1) 
when  P  is  greater  than  a  critical  value  c.  c  depends  only  on 
tie  *-atj.o  of  utilities,  ci2^C22'  si-nce 


'21 


c  +  c 
^12  ^21 


1+  (C12/C21) 


C 
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The  commander's  rating  depends  on  P  and  c,  with  P  incorporating 
(not  necessarily  consciously)  observations  evaluated  by  his 
knowledge  and  experience  while  c  is  indicative  of  attitudes 
towards  risk  and  uncertainty. 

To  estimate  c  and  make  predictions  about  s  from  rating  r, 
various  circumstances  relating  to  degrees  of  information  about 
P  can  exist.  At  one  extreme  is  the  unlikely  condition  when  the 
commander's  P  is  known.  A  more  likely  case,  requiring  more 
investigation,  occurs  when  the  commander's  assessment  of  P  is 
related  to  the  observations  of  the  ship.  Finally,  a  very  useful 
and  reasonable  model  results  when  something  is  known  about  the 
distribution  of  P. 

In  the  usual  decision  theory  situation,  there  is  a  known 
distribution  of  observations  for  each  state  of  nature.  Based 
on  these  distributions,  the  a  priori  probability  for  each  state 
of  nature  is  revised  to  a  new  a  posteriori  probability  for  each 
state  after  each  new  observation  or  group  of  observations.  This 
situation  is  not  exactly  present  in  our  case  because  the  relation¬ 
ships  between  the  observations  and  the  states  of  nature  (actual 
states  of  readiness)  are  very  complicated  and  are  not  sufficiently 
known  or  defined. 

P's  and  r's  Known  For  n  Occasions: 

In  this  situation,  the  data  for  n  occasions  consists  of 
(Pi ,r. ) ,  (P~,r~) , . . . , (P  .r  ) .  Since  for  r  «  2,  P  <  c  and  for  r  «  1 
P  >c,  we  obtain  bounds  P<  c  <  P,  where  P  is  the  maximum  of  the 
P's  associated  with  r =  2  and  P  is  the  minimum  of  the  P's 
associated  with  r-1.  The  bounds  P<c<P  can  also  be  translated 
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to  give  bounds  for  W19/W99  as  <  1  ~  -  .  This  case 

l4£  **  P  w22  P 

has  mathematical  features  similar  to  those  in  the  revealed 
preference  approach  in  economic  theory. 


Distribution  of  P  Known: 

In  a  number  of  circumstances  it  is  reasonable  to  assume  that 
some  knowledge  of  the  distribution  of  P  is  available.  This  infor¬ 
mation  can  sometimes  be  obtained  by  statistical  studies  of  the 
actual  ratings  of  a  number  of  ships  on  a  number  of  maneuvers. 
Suppose  the  probability  density  function  of  P  is  denoted  by  f(p). 

It  is  also  indicated  later  that  any  assumption  about  f (p)  can  be 
tested  statistically  by  means  of  suitable  data. 

Possible  forms  of  f(p)  are  the  uniform  distribution, 
various  kinds  of  triangular  distributions  and  the  Beta  distribu¬ 
tion  family  indexed  by  two  parameters. 

The  probability  that  the  commander  rates  the  ship  as 
satisfactory  (r=l)  is 


Prob  (r  =  1) 


J  f (p) dp  . 
c 


The  probability  of  r =  1  can  be  estimated  by  the  fraction,  g, 
of  satisfactory  ratings  on  n  occasions.  When  function  f (p)  is 
known,  c  can  then  be  estimated  by  6  from 

1 

g  =  |  f  (p) dp  . 
c 

For  example,  when  f (p)  is  the  uniform  distribution,  f (p)  =  1  for 
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(0£p£l)  then 

g  =  1  -  c  giving  c  *  1  -  g  . 

For  the  triangular  distribution  f(p)*2(l-p)  the  resulting 

estimate  c  :  is  obtained  from 

1 

g  =  |  2  (1  -  p)dp  =  (1-S)2 
c 

giving  estimate  c  -  1  -  /g  . 
c  Known: 

When  c  is  also  known ,  this  model  can  be  developed  to 
study  how  the  probability  that  the  actual  performance  is  satis¬ 
factory  (s  =  l)  relates  to  the  commander's  ratings. 

Without  the  commander's  rating, 

1 

Prob(s=*l)  =  |  Prob  (s  =  1  |P  =  p)f  (p)dp 
o 

1 

*  f  pf  (p)dp  =  E(P)  . 

J 

o 

With  the  commander's  rating,  say,  r  =  1,  the  conditional 
probability  is, 

Prob(s  *  1  |r  =  1)  =  Prob (s  *=  1  |P  >  c) 

1 

|  pf(P)dp 

c 
1 

J  f(p)dp 

c 


E  (P  |  P  >  c) 
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To  illustrate,  when  f (p)  is  the  uniform  distribution. 


Prob(s  *  1)  *  E(P)  »  j 


and. 


Prob(s  -  l|r  =  1)  =  E(P|P>c)  *  j  (1+c) 

Prob(s  =  1 1 r  =  2)  -  E(P|P<c)  -  jc  . 

When  f(p)  and  c  are  known,  the  joint  distribution  of  s 
and  r  can  be  derived.  For  instance,  again  using  the  uniform 
distribution  as  an  illustration,  the  joint  and  marginal  distri¬ 
butions  are 


Marginal 


1 

2 

1 

i 

Marginal 

1  j(l-C2) 

1  „2 

Ic 

|  _ 

j 

1 

1  2 

I 

j 

1 

1 

2  1  -  c  -  |(1  -  c2)  j 

_  i 

-i'1 

i 

i 

i 

i 

1 

2 

j 

i 

ial  1  -  c 

C 

I 

1 

i  i 

From  the  joint  distribution,  correlations  and  conditional 
distributions  can  be  calculated.  For  example. 


Prob(s  *  l|r  «  1) 


Prob(s  ■  l,r  » 1) 
Prob  (r  »  1) 


j(l-c2) 
1  -c 


j(l  +  c) 
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Assumptions  regarding  f(p)  and  c  can  be  tested  statisti¬ 
cally  from  data  of  commander's  readiness  ratings  and  of  actual 

states  of  readiness  on  n  occasions  (r,,s, ) , (r«,s~) , . . . , (r  ,s  ) • 
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For  example,  Prob(s  =  l,r  =  1)  is  estimated  by  the  fraction  of 
the  n  occasions  in  which  3  =  1  and  r  =  1. 

The  model  can  also  ie  used  to  predict  actual  performance 
using  the  commander's  ratings  and  to  study  how  good  these 
predictions  can  be  expected  to  be. 

The  various  models  can,  of  course,  be  studied  when  more 
than  two  rating  levels  are  us  ad,  giving  rise  to  multivariate 
distributions. 

In  addition,  if  the  ratings  of  different  factors  and  missions 
are  incorporated,  as  is  done  in  our  general  model,  it  is  very 
likely  that  better  prediction  procedures  can  be  developed. 


